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Abstract 

Since 1984, a low-level effort has been underway at Rocketdyne, 
manufacturer of the Space Shuttle Main Engine (SSME) , to automate 
much of the analysis procedure conducted after test firings. 
Previously published articles in corporate publications, 
international conference proceedings, and the international trade 
press have contained the context of and justification for this 
effort [4, 8, 9], and technical issues regarding the integration 
of large data bases with the run-time component of the 
knowledge-base system [2]. In this paper, progress is reported 
in building the full system, known as SCOTTY, after a noted 23rd 
century rocket propulsion expert. 

The progress is on two fronts. The first is an organizational or 
philosophical one: the automated analysis of SSME tests is a 
complex process, but only part of it typically involves an 
expert. That is, this expert knowledge-based system is called 
only when required; there are many more mundane tasks which may 
interface to an expert, but do not require heuristic technical 
expertise. Don't make your technical expert also serve as the 
program manager! 

The second front of progress is a technical one. Since the very 
inception of the program, it has been strongly believed that the 
intrinsic nature of SSME test analysis and character of 
inductive-based expert system building tools (ESBTs) represent an 
excellent match of problem and tool. It was, in fact, the 
driving consideration for the 1984 recommendation given to 
management, along with the critical feature of being software- 
compatible with existing systems, i.e., the tool must be able to 
generate Fortran code. The intuition has been justified by the 
relative ease with which a significant source of corporate 
diagnostic and analytical expertise has been transformed to 
examples and thence to effective production rules. The source of 
this expertise is in completed SSME anomaly forms, one for each 
of the 1400+ tests conducted since 1975. The latter 
transformation from examples to production rules is accomplished 
automatically with a powerful inductive ESBT, ExTran 7 from 
Intelligent Terminals Ltd. [1], running on a Concurrent Computer 
Corporation 3260 super-minicomputer. The engineering staff 
responsible for building (and eventually maintaining) SCOTTY has 
consistently used examples as input — a single rule has yet to 
be written. The knowledge-acquisition "bottleneck" is thus much 
wider than for most previously-reported expert systems. 


203 



The moderate expansion of the former low-level efforts in 
constructing an automated test analysis procedure for the SSME 
will be discussed in this paper. The topics will cover 
qualitative and quantitative details of the above organizational 
and technical issues, as well as various possible extensions 
being considered. These include: 

the integration of a large-scale relational data base system 
[3] and example generation at the knowledge acquisition 
component of ExTran 7 , 

a graphics interface for experts and end-user engineers, 

increased efficiency from exploiting concurrency in problem 
and machine, 

potential extension of a tuned and limited subset of the 
system to flight engines, 

application of the system for training purposes of many 
newly-hired engineers, 

incorporation of design and monitoring tasks into an 
automated system, 

technology transfer to other engines and Rocketdyne programs, 

an anomaly description language, and 

the essential qualities of good software engineering 

practices for building expert knowledge-based systems. 

Introduction 

Every time a Space Shuttle Main Engine (SSME) is test fired, 
hundreds of measurements are taken from a wide variety of 
sensors. Many more values are also calculated. All of these data 
values, when combined with previous performance of the engine and 
its components, are used by the engineering staff at Rocketdyne 
to determine the future tests. These outcomes can vary from all 
requirements being met, to a few minor events, to a rare 
significant event. As the SSME is the world's most complex 
reusable liquid-fuel (oxygen and hydrogen) rocket engine, 
Rocketdyne and NASA, the customer, consistently insist that a 
thorough investigation of each test firing be performed by our 
most highly-trained staff. This emphasis on quality is 
heightened even further as a result of the Challenger tragedy, 
ensuring that the next scheduled shuttle flight, Discovery in 
June of 1988, will be the safest that is humanly possible. The 
recent increase in the SSME testing schedule, to about twelve per 
month, is witness of this concern. 

To continue its virtually perfect record of supporting 
shuttle flights, Rocketdyne is always looking for ways, both 
technical and organizational, to improve the quality of our 
product while working within customer guidelines. One of the 
major methods involves making the most accurate diagnosis, 
analysis, and recommendation possible for the the next engine 
test or shuttle flight. To perform this task, reliance has been 
on maximal use of sophisticated tools and the expertise of an 
engineering staff. This staff has accumulated experience dating 
back to 1975 and covering 1400+ SSME firings, plus numerous other 
ones from the Apollo F— 1 and J— 2 engines to those on the Atlas. 

In addition to gaining incalculable experience over the 
years, the engineering staff has also been gaining an increase in 
another quality — age. Like most other aerospace companies, 
Rocketdyne has a significant gap between staff with 20-30 years 
of experience and those with 5-10 years. Although the young 
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engineers are bright and motivated, they are keenly aware they do 
not possess the wealth of background of our older senior staff. 
These staff engineers are approaching or at retirement age, but 
their replacements have considerably less rocket engine 
experience. Hence, Rocketdyne is confronted with a significant 
dilemma: how to improve the quality of SSME test analysis in the 
face of diminishing senior staff. Several options to solve this 
dilemma were discussed in [4]. It was decided to use a 
combination of staff, results from previous SSME tests, and 
automated tools to address the problem and begin to build a 
prototype for automated corporate expertise. 

Rocketdyne is far from alone in being confronted with the 
above problems. Indeed, the corporation has ample "company" in 
deciding to use a type of automated tool know as expert systems, 
part of the artificial intelligence technology. In fact, there 
is considerable concern that, once again, this field is in danger 
of being "over-hyped" [6], And the company is certainly not the 
first to decide to concentrate initially on a diagnosis type 
application, a type currently of considerable importance in 
industry despite being "old-hat" to the AI research community. 

So what is unique about SCOTTY, our automated system? There are 
two unusual aspects. 

One such aspect is the incorporation of SCOTTY as "another", 
albeit advanced, software tool which must: 

live in a distributed corporate environment, 
talk to large data bases, 

be maintained by existing engineering staff, 
run with color graphics terminals, 
execute on standard computers, 

be amenable to parallel processing hardware, and 
meet corporate-wide software engineering development and 
quality guidelines. 

The other unusual aspect is a technical one which increases 
the ease with which SCOTTY can be constructed. By use of a type 
of ESBT known as inductive or example-based, the historical 
expertise now reposing in the anomaly sheets for the hundreds of 
SSME tests can be transformed into examples, and thence 
automatically into production rules. These rules will, in turn, 
drive SCOTTY during normal day-to-day operation in future years. 

SCOTTY 

History 

In 1984, the author was hired by Rocketdyne to assist in the 
construction of an automated tool for SSME test analysis. Within 
two months, a proof-of-concept model for a High Pressure Oxidizer 
Turbo Pump (HPOTP) had been built. This involved recommendation 
of an inductive ESBT, Expert Ease by Intelligent Terminals, Ltd 
(ITL) in Glasgow, Scotland, and the first such PC-based ESBT 
commercially available. The tool was purchased and used, after 
minimal training time, by a mechanical engineer, to diagnose 
HPOTP anomalies, by specifying 42 examples and nine attributes. 

A 48 rule subsystem was automatically generated by Expert Ease. 

No rules were required of the engineer. This prototype and the 
problem context, rationale, and solution were described in an 
early paper [4]. A desirable tentative system configuration is 
shown in Figure 1. 
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Figure 1. SCOTTY: Figure 2. Building Expert Systems 

Final System Configuration with ExTran 


During 1985 and 1986, the system (now named SCOTTY) underwent 
several extensions. From a tool viewpoint, a more powerful ESBT 
became available. ExTran 7, an industrial strength Fortran- 
based inductive ESBT from ITL which runs on a wide variety of 
machines from PCs to workstations to super-minis to mainframes, 
was recommended. A process for using ExTran is given in Figure 
2 . ITL ported the product to the available Concurrent Computer 
Corporation 3260 super-mini at minimal cost. The HPOTP examples 
were immediately transported to ExTran and the resulting module 
was now a true, albeit simple, knowledge base system (KBS) 
utilizing "Why", "How", and "What if" type questions, history 
files, external interfaces, and all the other features usually 
associated with a KBS. 

Conceptually, SCOTTY was extended in several directions 
durin 9 this same time period. It was demonstrated that multiple 
problems could be run concurrently on the multiple processor 
Concurrent 3260. Graphics routines (PLOT-10 and GKS libraries) 
were tied to ExTran with a minimum interface. In-house 
statistical routines were easily linked to SCOTTY. Small Fortran 
routines were written to access SSME test files and output 
attribute values for input to SCOTTY sub-problems. Additional 
SSME component modules were specified. Figure 3 contains the 
early version of a structure chart. A major extension was the 
run-time interface between ExTran and the large data base 
managment system DMS/32 supplied by Concurrent (then known as 
Perkin-Elmer) . These are all described extensively in a paper 
presented in 1986 [2]. 
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Figure 3. SCOTTY Structure Chart, Version .84 


Current Status 

SCOTTY now exists at a stage between a research prototype and 
a production model, using the taxonomy of Waterman [6]. It is 
not being used in production now — additional resources would 
permit this to occur sooner. SCOTTY consists of far more than 
"just" an expert system, as is clearly shown in figure 4, but 
rather is one component in a fairly extensive software system. 
This reflects the strong belief that viable expert systems are 
most likely to succeed in a hybrid and integrated environment, 
where they must communicate easily with other standard existing 
and future sub-systems. 



Figure 4. Context of SCOTTY (Automated Test Data Expert) 

An updated structure chart reflects the understanding that 
SSME test analysis involves several levels of expertise, from 
relatively mechanical but comprehensive data review to component 
and system level diagnostic and analytical experts. See Figure 
5. For many routine tests, the expert system is not required. 
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This is no stranger than the human equivalent case of calling on 
resources only when needed. The program manager decides when 
he/ she must draw on the rare and expensive human expert. Do not 
make the mistake of having your senior technical specialist 
trying to "double” as program manager. These two entities have 
different types of expertise. 



Figure 5. SCOTTY Structure Chart, Version .9X 

SCOTTY, as of mid 1987, consists of 18 ExTran modules 
comprising 3200 lines of code (LOC) in Fortran. Supporting code 
required 5000 LOC. The ExTran generated code was derived 
automatically from approximately 260 examples. No rules have 
been written by hand to date — all 700 rules were induced. 


Extensio ns in Progress 

Development is continuing on a number of fronts for SCOTTY. 
Three are highlighted here: graphics, anomaly data, and a new 
extension to ExTran. 

As SCOTTY matures, the level of effort devoted to its 
development has increased. The first efforts were done solely on 
Rocketdyne internal R&D funds. Currently, funding efforts have 
been underway for some time with a customer for making the system 
a production one more quickly. Recently h ^ re ^. eng ^® e ff ^ 
computer scientists have been active in extending the structure, 
leaving the knowledge content to acknowledged experts. A SSME 
instrumentation chart, now taped to the walls of hundreds of 
Rocketdyne engineering offices, is being converted to a dynami 
color computer graphics form. See Figure 6 for a sample, 
purposefully simple, display of a Low Pressure 

(LPFTP) . This graphics subsystem will have capabilities to zoo , 
highlight problem areas (according to actual test data 
measurements) , and depict flow. This is not CAD/CAM, although 
there are a few common themes, nor is it extensive CFD mo e mg 
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of the National Aerospace Plane (NASP) using multi-million dollar 
CRAY 2s. It is a practical and feasible use of moderate color 
resolution on the readily available super-mini and terminals. 
Engineers on the floor, as would be expected, are very pleased to 
see in graphical form what they have hitherto had to dig out of 
static tables and plots. 



Figure 6. Simple Graphic Display for a LPFTP 

Anomaly data is a key to the success of SCOTTY. It provides 
a starting point for converting much of the SSME testing 
expertise repository into machine readable form. Serious efforts 
are underway to use this source to augment the experience now 
encapsulated in the heads of senior engineering staff. Each 
anomaly sheet consists of three major fields: problem (symptoms), 
analysis (causes) , and recommendations. See Figure 7 for a 
(dummy) sample. Zero or more anomalies are recorded for each 
test, usually very minor ones. By carefully reviewing each 
anomaly and any back-up plots/tables, it is possible to convert 
each one into an example format consisting of a set of 
attribute-values and decisions. Anomalies for the first several 
tests tried are converted rather slowly, as new attributes are 
frequently added. However, as experience in the conversion 
process is gained, and the rate of growth of new attributes 
slows, the rate of the anomaly to example format conversion 
increases significantly. 
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Figure 7. Sample Anomaly Data 
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The third extension underway is the intention to use a new 
feature of ExTran which is the result of a joint project between 
ITL and Concurrent with roots in the earlier work at Rocketdyne 
[2]. This feature extends the interface between ExTran and a 
data base system to the knowledge acquisition component of the 
former, as well as the run-time interface discussed. in [2]. See 
Fioures 8, 9, and 10. The effort of this joint project is known 
as Reliance-Expert, and is scheduled to undergo beta test at 
Rocketdyne. It would permit the anomaly data to be transformed 
to records in a relational DBMS. This would make this valuable 
data available for a wide variety of uses. One of the uses would 
be to serve as an "expert" for historical knowledge of SCOTTY, as 
it can now be transformed automatically into examples and then to 
rules. So, once again, the knowledge acquisition bottleneck 
becomes less and less of an issue, as it will be possible to go 
directly from anomaly records in a DBMS to production rules in an 
expert system. 



Reliance 


EXTRAN 

Relational Database 


Expert System 
Generator 






Reliance Expert 


Figure 8. Reliance 
Expert Structure 


Figure 9. Reliance Expert 

Development Phase 
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Future Development Issues 

Further in the future are several concerns. There is an 
interest in each as a potential contributor to improving the 
quality of SSME test analysis. Obviously, Rocketdyne is keenly 
concerned also about technology transfer to other types of 
engines, in addition to the SSME. The company is deeply 
committed to the Advanced Launch System (ALS) , Space Station 
power, National Aerpspace Plane (NASP) , Orbital Transfer Vehicles 
(OTV) , and other propulsion and energy systems. 

These further-reaching concerns are concentrated both in 
application and technical areas. On the application side, 
Rocketdyne would like to investigate the potential of extending 
SCOTTY to handle a limited subset of the measurement data for 
flight engines. The incorporation of monitoring and design tasks 
is also of high interest. An obvious application is to enlarge 
the context of SCOTTY to include new hire training on SSME test 
analysis. 

On the tool side, the issue of dealing with uncertain and/or 
noisy example data is significant. Real engineering problems 
involve uncertain and incomplete information. Indeed, a noted 
nuclear engineer, Dr. Billy Koen at the University of Texas in 
Austin, has gone so far as to define the engineering method as 
"the use of heuristics to cause the best change in a poorly 
understood or uncertain situation within the available resources" 
[5]. This feature exists on several commercial tools, but only 
in a pre-release form for ExTran, as of this date. The 
possibility of using abductive reasoning for diagnosis also 
appears to hold some promise [7], 

Conclusions 

Since 1984, a low-level effort has been underway at Rocketdyne, 
manufacturer of the Space Shuttle Main Engine (SSME) , to automate 
much of the analysis procedure conducted after test firings. In 
this paper, we reported on progress in building the full system, 
known as SCOTTY, after a noted 23rd century rocket propulsion 
expert . 

The progress is on two fronts. The first is an organizational 
one: the automated analysis of SSME tests is a complex process, 
but only part of it typically involves an expert. Don't make 
your technical expert also serve as the program manager! 

The second front of progress is a technical one. Since the very 
inception of the program, it has been strongly believed that the 
intrinsic nature of SSME test analysis and character of 
inductive-based ESBTs represent an excellent match of problem and 
tool. The intuition has been justified by the relative ease with 
which a significant source of corporate diagnostic and analytical 
expertise has been transformed to examples and thence to 
effective production rules. The source of this expertise is in 
completed SSME anomaly forms, one for each of the 1400+ tests 
conducted since 1975. The latter transformation from examples to 
production rules is accomplished automatically with a powerful 
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inductive ESBT, ExTran 7, running on a Concurrent Computer 
Corporation 3260 super-minicomputer. The engineering staff 
responsible for building (and eventually maintaining) SCOTTY has 
consistently used examples as input — a single rule has yet to 
be written. The knowledge-acquisition "bottleneck" is thus much 
wider than for most previously-reported expert systems. 


The moderate expansion of the former low-level efforts in 
constructing an automated test analysis procedure for the SSME 
was discussed. The topics covered qualitative and quantitative 
details of the above organizational and technical issues, as well 
as various possible extensions being considered. These included: 
the integration of a large-scale relational data base . system 
and example generation at the knowledge acquisition 
component of ExTran 7 , and 

a graphics interface for experts and end-user engineers. 
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